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ABSTRACT: 

Scientists with NASA’s Earth Science Applications Directorate are creating a well-characterized Verification & Validation (V&V) 
site at the Stennis Space Center (SSC). This site enables the in-flight characterization of remote sensing systems and the data that 
they acquire. The data are predominantly acquired by commercial, high-spatial resolution satellite systems, such as DCONOS and 
QuickBird 2, and airborne systems. The smaller scale of these newer high-resolution remote sensing systems allows scientists to 
characterize the geometric, spatial, and radiometric data properties using a single V&V site. The targets and techniques used to 
characterize data from these newer systems can differ significantly from the techniques used to characterize data from the earlier, 
coarser spatial resolution systems. Scientists are also using the SSC V&V site to characterize thermal infrared systems and active 
Light Detection and Ranging (LIDAR) systems. SSC employs geodetic targets, edge targets, radiometric tarps, and thermal 
calibration ponds to characterize remote sensing data products. This paper presents a proposed set of required measurements for 
visible- through-longwave infrared remote sensing systems, and a description of the Stennis characterization. Other topics discussed 
include: 1) use of ancillary atmospheric and solar measurements taken at SSC that support various characterizations, 2) other sites 
used for radiometric, geometric, and spatial characterization in the continental United S tates, and 3) the need for a standardized 
technique to be adopted by the Committee on Earth Observation Satellites (CEOS) and other organizations. 


1. INTRODUCTION 

NASA scientists and others who use remote sensing data to 
make policy, economic or scientific decisions should have 
confidence in and an understanding of its characteristics in 
order to make informed decisions about its use. Confidence in 
the data is gained by performing characterizations or data 
assessments. These assessments can be broken down into three 
categories: geometric, spatial and radiometric characterizations. 
The importance of each c haracterization depends on how the 
data will be used. Geometric characterization includes both 
relative and absolute location assessments as well as elevation 
assessments. Spatial characterization involves Ground Sample 
Distance (GSD) and a measure of image sharpness such as 
edge response (edge slope), line spread function or Modulation 
Transfer Function (MTF). Radiometric characterization 
includes both relative and absolute measurements. 
Understanding remote sensing data characteristics becomes 
particularly important to scientists at NASA as they begin to 
incorporate commercially obtained high-spatial resolution data 
from systems designed and operated outside of the NASA 
scientific community into their research. 

The NASA Stennis Space Center (SSC) in Mississippi is being 
developed as a remote sensing Verification & Validation 
(V&V) site. The site, also referred to as the Stennis Fee Area, 
is relatively square and encompasses approximately 55 sq km 
of Government-owned land. Land cover is predominately 
forested, but also contains a significant number of natural and 
man-made features including grasslands, marshes, canals, 
ponds, buildings, rocket motor test stands, roads, etc. 


Targets, ground truthing measurement and analysis techniques 
required to characterize the different aspects of a remote 
sensing system are being put in place at SSC. The techniques 
being developed at this site are examples of standardized 
measurement techniques. Because the spatial resolution of 
remote sensing systems continues to increase and the swath of 
these systems is often decreasing, it is now possible to measure 
the geometric, spatial and radiometric properties of a remote 
sensing system at a single site within a single acquisition. 
Although much of the literature focuses on radiometry, 
geometric and spatial characterizations are often more 
important in high-spatial resolution systems. SSC has a 
heritage of working with commercially acquired remote 
sensing data (NASA, 2002) and has already begun to evaluate 
IKONOS and QuickBird 2 imagery taken over the Stennis site 
(NASA/N2MA/USGS, 2002). 

This paper begins with a discussion of geometric and spatial 
characterizations and then follows with a discussion on the 
more traditional radiometric characterization methods. 


2. GEOMETRIC CHARACTERIZATION 

2.1 Geometric Characterization Site Requirements 

Geometric characterizations are performed with an array of 
“photo-identifiable” known points or targets located at the 
V&V site. For the errors associated with defining the location 
of the points to become insignificant to the characterization 
process, their position must be known to an order of magnitude 
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better than the spatial resolution of the system (Federal 
Geographic Data Committee, 1998). For example, to 
characterize a 1-m GSD system, the known points must be 
surveyed to within 10 cm. At this level of precision, unbiased 
errors are a few percent for a single measurement. The points 
must also be unambiguously identifiable in the remotely sensed 
imagery. To do this, the targets must have a significant 
contrast against their surrounding. 

To meet these requirements, an array of geodetic targets has 
been deployed at the SSC. These white-painted targets are 8 ft, 
or 2.44 m, in diameter and are shown in Figure 1 (left section). 
The centers of these targets are painted red and have a diameter 
of 2 ft, or 0.61 m. A red center was chosen to better identify 
the targets on film-based systems, and plays no role in the 
characterization of digital sensing systems being discussed. 

In addition to the geodetic targets, targets of opportunity such 
as manhole covers are being used to perform geometric 
characterizations. One such target is shown in Figure 1 (right 
section). Currently, 136 manhole covers, ranging from 0.637 m 
to 2.9 m in diameter have been painted and surveyed at Stennis. 
White paint with nominally a 60 percent reflectance value in 
the visible spectrum was selected. 


Figure 1 . Geodetic target and painted manhole cover target at 
SSC 

The geodetic targets have been deployed evenly throughout 
SSC as shown in Figure 2. There are 44 geodetic targets 
deployed and plans are in place to deploy an additional 45 
targets during the coming year. Figure 2 also highlights the 
location of the manhole cover targets. These targets are found 
along roads within the site, and predominately generate long 
target arrays in the E-W and N-S directions. 
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Figure 2. Target positions throughout SSC fee area 


All of the targets discussed have been surveyed using standard 
static surveying techniques. The center of each target is known 
to within 6 cm horizontal and 9 cm vertical accuracy. Periodic 
checks are made to ensure that these known point locations 
remain stable over time. To assist in the surveying of these 
targets, 17 A-order monuments exist within the SSC Fee Area 
and are also shown in Figure 2. A National Geodetic Survey 
(NGS) Continuously Operated Reference (COR) site, 
maintained by the National Oceanic and Atmospheric 
Administration National Data Buoy Center (NOAA NDBC), 
provides carrier phases and code range measurements and is 
also within the SSC Fee Area. 

2.2 Geometric Characterization Techniques 

An automated process has been developed at SSC to evaluate 
the position of the targets on remotely sensed imagery as 
compared to known surveyed target locations. Commercially 
available software that displays the location of the 
georeferenced imageiy is utilized. 

2.3 Other Geometric Characterization Sites 

SSC maintains a target-rich environment from which to 
characterize the geometric properties of remotely sensed data. 
The site however, has a fairly constant elevation. Sites such as 
Morrison Quad in Colorado and Kaintuck Hollow Quad in 
Missouri are characterized sites that offer more Earth surface 
relief. They are maintained and used extensively by the U.S. 
Geological Survey (USGS). 

3. SPATIAL CHARACTERIZATION 

3.1 Spatial Characterization Site Requirements 

One way to perform spatial characterizations at a V&V site is 
to measure a sensor’s ability to image an edge. Edge targets, 
which are high contrast areas that form a sharp edge from a 
uniform dark area to a uniform bright area, are used in this type 
of characterization. The contrast between the dark and bright 
areas should strive to maximize the Digital Number (DN) range 
of the sensor being evaluated to maximize the sensor’s signal- 
to-noise ratio (SNR). When imaged, at least 30 pixels should 
lie on the target: 10 on the bright area, 10 on the dark area and 
10 along the transition or edge (Shea, 1999; Reichenbach et al., 
1991). Edges should also be formed in two planes 
perpendicular with each other so that edge analysis in two 
directions can be performed simultaneously. 

At the SSC there are two different types of edge targets that are 
currently being employed to evaluate the spatial properties of 
an image product. The first type of edge target consists of an 
edge pattern painted on a concrete slab. The pattern and 
dimensions are highlighted in Figure 3. The same pattern is 
painted twice, rotated 3.7 degrees apart, to enable digital 
imagery having different image coordinate systems, the ability 
to provide the appropriate number of pixels across an edge. 
Each pattern contains edges in both a northerly and easterly 
direction. The targets are large enough to evaluate imagery 
having GSDs up to 1 meter. The dark areas were painted with 
paint that is nominally 5 percent reflective in the visible 
spectrum, and the bight areas were painted with paint 
nominally 50 percent reflective. 

The second type of edge target employed at SSC is formed 
using two contrasting radiometric tarps, manufactured by MTL, 






as shown in Figure 4. The advantage these tarps have over the 
painted concrete targets is that they can be deployed at alternate 
sites. Each contrasting color of the radiometric tarps is 20-m 
by 20-m in size. Two different contrast edges can be formed 
with the tarps: 3.5 to 52 percent reflectance and 22 to 34 
percent reflectance. These tarps are large enough to 
characterize the spatial characteristics of 1-m GSD class 
imagery. 





Figure 3. SSC concrete edge target layout 







Figure 4. Tarp edge target 

The tarps were manufactured under very strict specifications. 
Each seam is straight to within 6 cm along a 20-m length. The 
standard deviation about the average reflectance does not vary 
by more than 1 percent spatially. Peak to peak variation in 
reflectance is less than 10 percent within any 100-nm spectral 
band and there is less than 10 percent variation in reflectance 
values when measuring the tarps from 10 to 60 degrees off 
axis. 


3.2 Spatial Characterization Techniques 

The technique used at SSC to evaluate the spatial performance 
of edges in imagery begins with deploying the target edge such 
that it is tilted between 4 and 8 degrees from the image pixel 
frame of reference. This enables the analyst to generate an 
appropriately sampled edge with at least 10 pixels defining the 
edge. 

Once the edge, properly oriented in the imagery is obtained, the 
edge response, measured in Digital Numbers (DN), is plotted as 
a function of distance from the known edge. A best fit through 


these points is then generated. The resulting plot is known as 
the system edge response and is shown in Figure 5 (top 
section). The derivative of this plot, called the line-spread 
function, is then calculated. Quite often the analysis ends with 
calculating the Full Width at Half Maximum (FWHM) of the 
line-spread function as shown in Figure 5 (center section). This 
parameter gives a measure of how sharp an edge appears in the 
image; the smaller the value, the steeper the edge response, and 
the sharper the image appears. 

Often scientists quantify an image’s spatial characteristics with 
a single parameter known as the MTF at Nyquist frequency. 
This parameter is generated by looking at the value of the 
Fourier Transform of the line-spread function at half the 
sampling frequency (1/(2* GSD)), shown in Figure 5 (bottom 
section). The higher the value, the sharper the image, but the 
greater the aliasing effects. 



Figure 5. MTF calculation 

It should be noted that this edge response technique is highly 
dependent on how accurate the edge response is defined. Small 
errors induced by curve fitting the edge response can lead to 
FWHM errors in excess of 10 percent. The errors are carried 
over into the MTF at Nyquist calculation. This method is also 
highly sensitive to the uniformity of the bright and dark areas 
of the edge target. Error analyses are currently underway at 
SSC. 


3.3 Other Spatial Characterization Sites 

Historically, spatial characterization of relatively large GSD (> 
30 m) imagery has been accomplished by imaging very large 
man-made edges such as bridges or buildings. South Dakota 
State University (SDSU) generates edges by deploying tarps in 
a similar fashion as currently described at their university site. 

In addition, commercial vendors of high spatial resolution 
imagery have constructed and maintain their own proprietary 
edge targets within the U.S. 




4. RADIOMETRIC CHARACTERIZATION - 
REFLECTIVE REGION 

4.1 Reflective Radiometric Characterization Site 
Requirements 

A vicarious radiometric characterization, in the context of this 
paper, predicts aKsensor radiance by measuring the reflectance 
of a ground target, modeling the atmosphere using a radiative 
transport code and measured atmospheric properties, and then 
propagating the ground target radiance through the modeled 
atmosphere. To perform these characterizations, a V&V site 
must possess the ability to accurately measure the reflectance 
of a ground target and appropriate properties of the atmosphere 
between the target and the sensor at the time a scene is imaged. 
The target can be either man made or a naturally occurring 
feature such as sand or grass. The target must have a relatively 
uniform reflectance such that the pixels describing the target do 
not vary considerably from one to another. Atmospheric 
properties that control radiative transport in the reflective 
spectral region must be measured as well. This includes down- 
welling solar irradiance, aerosols, water vapor and atmospheric 
pressure. 

Several ground targets are used to make radiometric 
characterizations at the SSC V&V site. The 20-m x 20-m 
radiometric tarps discussed above are the primary targets used 
to evaluate imagery having GSDs that are 4 m or less. These 
tarp targets can be deployed individually, in pairs to form 
edges, or all together in a checkerboard pattern. In addition to 
the tarps, a relatively uniform grass field and a portion of a 
concrete parking lot are often used as radiometric targets. An 
image generated by the Space Imaging IKONOS sensor shows 
a typical target layout at SSC (Figure 6). Two of the tarps have 
been aligned to form an edge, so that they can be used in both 
spatial and radiometric characterizations. A third radiometric 
tarp is to the left. Two smaller tarps, which act as fiducial 
markers, mark the grass target area further to the left. A closer 
inspection of the image reveals two additional small black 
fiducial markers, identifying a 20-m x 30-m concrete area in 
the lower left-center portion of the scene. 
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Figure 6. IKONOS image of tarp layout at SSC 

The SSC V&V site maintains several Analytical Spectral 
Devices (ASD) spectroradiometers for measuring the 
reflectance of ground targets. A sensor calibration lab at SSC 
calibrates the spectroradiometers and much of the ancillary 
equipment necessary to make these measurements. 


Several instruments are used at the SSC to measure 
atmospheric properties. Stennis maintains two different solar 
radiometers as shown in Figure 7. The Automated Solar 
Radiometers (ASRs, left side), are manufactured by the 
University of Arizona (UofA) (University of Arizona, 2002) 
and the Multifilter Rotating Shadowband Radiometers 
(MFRSRs, right section), are manufactured by Yankee 
Environmental Systems (Yankee Environmental Systems, 
2000). During a typical ground truthing campaign, several 
solar radiometers are deployed to measure downwelling solar 
irradiance in select narrow bandwidths, and data from each 
instrument is compared. In addition, a Radiosonde weather 
balloon is launched within a half hour of when the sensor 
acquisition takes place. A Radiosonde launch provides 
temperature, pressure and relative humidity measurement 
profiles of the atmosphere. 



Figure 7. Solar radiometers at SSC 

4.2 Reflective Radiometric Characterization Techniques 

At the time of the sensor acquisition, radiometric target 
reflectance values and atmospheric properties are measured. 
Typical reflectance measurements from a radiometric 
characterization are shown in Figure 8. Thousands of spectra, 
taken by more than one instrument to reduce the risk of 
instrument bias or failure, are typically obtained. 



Figure 8. Measured tarp reflectance values 

Atmospheric transmission values in several bands are 
calculated based on the solar radiometer measurements. The 
values, along with other key parameters such as measured 
atmospheric water vapor content, barometric pressure, position 
on Earth, and time of image acquisition are used to formulate a 
model atmosphere. NASA Stennis scientists use the 
MODTRAN radiative transport code (Berk et ah, 1999) to 
establish this model atmosphere. A typical model atmosphere, 
with its comparison to measured values, is shown in Figure 9. 

The model atmosphere is then coupled with the measured target 
reflectance values to predict at-sensor radiance values. These 
predictions are compared to image-acquired values to evaluate 
the radiometric performance of a sensing system. 




Figure 9. Atmospheric transmission 


4.3 Other Reflective Radiometric Characterization Sites 

The SSC is unique in that this site is capable of providing 
several different radiance values spanning a significant portion 
of a sensor’s DN range. It should be noted however, that 
weather constraints hinder the year round utilization of this site. 
There are several playas in the western United States that have 
been used by NASA and the UofA to make comparable 
measurements. These include Lunar Lake playa in Nevada, 
Railroad Valley playa in Nevada, Ivanpah playa in California 
and the White Sands Missile Range in New Mexico. These 
sites are highly desirable because of their high reflectance, high 
elevations and relatively benign weather, even though they are 
fairly remote and difficult to access. Other more challenging 
sites include the SDSU V&V site, whereby grass fields are 
used as radiometric targets. 


5. RADIOMETRIC CHARACTERIZATION - 
THERMAL REGION 

5.1 Thermal Radiometric Characterization Site 
Requirements 

Thermal characterizations are often performed using large 
uniform water bodies. These water bodies typically need to be 
at least 20-30 ft deep to ensure uniformity and encompass at 
least 10 pixels spatially. Using water bodies allows one to take 
advantage of the high emissivity of water. Approximately 99 
percent of the radiation in the thermal region is absorbed in the 
first 1 00 pm of water. Therefore, only the v ery thin layer of 
water at the water-air interface, commonly referred to as the 
skin, radiates. In this way, the most accurate way to measure 
the temperature of a water body is to make radiant temperature 
measurements. 

Just as in reflective region radiometric characterizations, the 
atmosphere at the time of a sensor acquisition must be 
measured. Aerosol concentrations are not as significant in this 
part of the spectral region. Atmospheric water vapor 
concentration, temperature and pressure, however, become 
critical. 

At SSC a man made High Pressure Industrial Water Reservoir 
(HPIWR), primarily used to provide a water source to cool the 
rocket engine test stands also on site, is used as a water body 
source. This reservoir is shown in Figure 10. 


Figure 10. HPIWR at SSC 

5.2 Thermal Radiometric Characterization Techniques 

At the time of the sensor acquisition, water body radiant 
temperature measurements and atmospheric properties are 
measured. To measure the radiant temperature of a body of 
water, NASA Stennis scientists designed and built a thermal 
measurement float, which is shown in Figure 11 (Pagnutti, 
2002). The float takes radiant temperature measurements with 
Heimann radiometers p ointed a t b oth t he w ater b ody a nd the 
sky. In addition, physical temperature measurements are made 
with a thermocouple probe at several depths b elow the water 
surface. Atmospheric information such as wind speed, wind 
direction and air temperature are measured as well. 



Figure 1 1 . SSC Measurement Float 

The SSC V&V site maintains several radiometers and black 
bodies for measuring the radiant temperature of ground targets. 
The sensor calibration lab at Stennis calibrates these and other 
ancillary equipment necessary to make these measurements. 

A Radiosonde is also launched within a half hour of when the 
sensor acquisition occurs to generate atmospheric profiles of 
temperature, pressure and water vapor content. Just as in the 
reflective radiometric characterization technique, NASA 
Stennis scientists use the MODTRAN radiative transport code 
to establish a model atmosphere, based on the radiosonde data 
collected. 

The model atmosphere is then coupled with the measured water 
body temperature to predict at-sensor radiance values of the 
water body. These predictions are compared to image-acquired 
values to evaluate the radiometric performance of a sensing 
system. 

5.3 Other Thermal Radiometric Characterization Sites 

Thermal radiometric characterization sites are typically 
centered on large, deep-water bodies. A site used by many 
NASA scientists is at Lake Tahoe. The NASA Jet Propulsion 
Laboratory (JPL) maintains four thermal measurement floats 
there. (Hook, 2000). 




6. ACTIVE LIDAR CHARACTERIZATION 

To perform LIDAR characterizations, a well known, surveyed 
site must exist. Not only must features be identified, but their 
relative and absolute positions must be known. As with the 
geometric characterizations, these features must be located to 
an order of magnitude better than the resolution of the LIDAR 
product. In addition, if intensity based LEDARs are to be 
characterized, edge responses and biases with reflectance will 
be characterized. 

To meet these requirements, SSC scientists have generated an 
accurate digital elevation model (DEM) of the site. This model 
is based on aerial photography flown over the site and scanned 
at 6-in resolution. The imagery was georectified using 26 
ground control points surveyed to an accuracy equal to or better 
than 16 cm. The model was generated using the commercially 
available Socet Set software (BAE Systems, 2001). 

6.1 LIDAR Characterization Techniques 

An automated process to evaluate LIDAR data is currently 
planned at SSC. This process will evaluate the positions of 
known features and compare them to LIDAR data products. In 
the case of LIDAR intensity images, locations of known 
contrast will be incorporated. Of particular value will be the 
edge targets described above. 

6.2 Other LIDAR Characterization Sites 

Manufacturers of LIDAR systems quite often characterize them 
at their own test sites. Other groups such as the University of 
Florida along with NOAA and the Federal Aviation 
Administration (FAA) have done extensive LIDAR evaluations 
(Woolard, 2001). 


7. CONCLUSIONS 

The NASA Stennis Space Center in Mississippi is being 
developed as a remote sensing V&V site. Several different 
targets as well as the ground truthing measurement and analysis 
techniques required to characterize the performance of a remote 
sensing system are being put in place. This site is unique in 
that it is capable of simultaneously characterizing the 
geometric, spatial and radiometric performance of high-spatial 
resolution remote sensing systems. It is also being developed 
as a L IDAR c haracterization site. The site is currently b eing 
used to characterize commercially acquired imagery from 
IKONOS and QuickBird 2. 

The CEOS and other organizations need to adopt a 
standardized technique to characterize remotely sensed data of 
all scales. The V&V site at SSC begins to address that need. 
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